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tie: Clock Offset Estimation with Bias Correction 
d of the Invention 

The present invention relates to synchronization of clocks. More specifically, the 
present invention relates to estimation and correction of clock offset in distributed 
resources interconnected by a network, such as the Intemet. 
Background of the Invention 

Accurate and reliable time information is necessary for many systems and 
applications involving distributed resources, including networked systems and processes 
typified by the Intemet, In such systems, different functional elements are required to 
have their clocks synchronized. Clock synchronization involves two aspects: frequency 
synchronization and time synchronization. The former means that element clocks run at 
the same frequency, and the latter means that elements agree at a particular epoch with 
respect to the coordinated universal time (UTC), ie., there is no offset between element 
clocks. For many purposes, it is appropriate to focus on estimating clock offset and to 
assume higher order effects, such as the frequency offset, can be ignored or provided for 
separately. 

Clock synchronization issues have been extensively addressed in the literature. 
See, for example, D, Mills, Intemet time synchronization: the Network Time Protocol, 
IEEE Trans. Communications, Vol. 39, No.lO, October 1991; D. Mitra, Network 
synchronization: analysis of a hybrid of master-slave and mutual synchronization, IEEE 
Trans, Communications, COM-28, 8 (August 1980), pp.1245-1259; andN.W. Rickert, 
Non Byzantine clock synchronization - a programming experiment, ACM Operating 
Systems Review 22,1 (January 1988), pp.73-78, 

A well-known clock synchronization protocol that has been successfully deployed 
in the Intemet is the Network Time Protocol (NTP), described, for example, in D. Mills, 
Network Time Protocol (version 3) Specification, Implementation and Analysis RFC 
1305, March 1992. One of the most important network clock synchronization issues 
addressed by NTP is how to use the collected data to estimate the clock offset between a 
pair of network elements. 

In V, Paxson, On Calibrating Measurements of Packet Transit Times, LBNL- 
41535, ftp://ftp.ee.lbl.ROv/papers/vp-clocks-sigmetrics98.ps.gz , March, 1998 (and in a 
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shortened paper with the same title published in Proa ACM Sigmetrics98, June 22-26, 
1998), the author proposed a new algorithm for clock offset estimation. For easy 
reference, this algorithm will be referred to as the Separate Direction Estimation 
Algorithm (SDEA). While SDEA can provide improved performance relative to the 
5 NTP algorithm, SDEA nevertheless suffers from significant limitations, especially in 
applying SDEA to contexts in which loading is different for each of the directional links 
between pairs of network elements. 
Summary of the Invention 

Limitations of the prior art are overcome and a technical advance is made in 
10 accordance with the present invention, illustrative embodiments of which are presented in 
the following detailed description. In particular, limitations of the NTP algorithm and 
SDEA are overcome and SDEA techniques are extended and improved in accordance 
% with present inventive teachings. 

H In addition to estimating the clock offset, present inventive techniques also 

1:5 15 estimate the bias of the estimation and attempt to correct any such bias. As a result, 
Jp present inventive algorithms show significant improvement in terms of convergence 

fU speed and accuracy. Illustrative embodiments of present inventive algorithms will be 

m referred to as Separate Direction Estimation Algorithms with Bias Correction 

f (SDEABC). 

fij 20 In accordance with one aspect of present inventive methods, messages are 

2 exchanged (bi-directionally) between pairs of network elements, such messages including 

timestamps indicative of sending and receiving times noted at each stop. Because 
variable components of delay for each message direction need not be characterized by 
identical probability distribution functions, e.g., when links in each direction are 
25 differently loaded, undesired bias of estimates for clock offset can emerge. Illustrative 
embodiments of the present invention avoid errors in estimates of variable delay 
minimums by separately determining these minimums, e.g., by advantageously 
employing separate empirical probabiUty distributions for each direction. 
Brief Description of the Drawing 
30 FIG. 1 illustrates time relations in sending messages between network elements. 
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FIG. 2 shows an illustrative network embodiment of the present invention having 
a single server node for interacting with a plurality of other nodes to effect clock 
correction at such other nodes. 

FIG. 3 shows an illustrative network embodiment of the present invention having 
5 a hierarchy of server nodes with lowest order server nodes for interacting with other 
network nodes to effect clock correction at such other nodes. 
Detailed Description 
Separate Direction Estimation Algorithm 

In network clock synchronization, each of a pair of network elements exchanges 
10 data packets {timing messages) with the other of the pair. One of such a pair is referred 
to as a sender, and the other as a receiver with respect to a particular packet. Based on 
time stamps contained in these timing messages, the clock offset between the sender and 
the receiver is estimated. 
^4 More specifically, timing messages are sent between network elements as shown 

5J 15 in FIG. 1, where activities at a sender 100 and a receiver 110 are shown for a particular 
f| round of message exchanges. At the /-th round of message exchange, the /-th message 

lU includes a time stamp indicating current time as known at the sender 100 when the 

;:|t 

P message is sent to the receiver 1 10. Immediately upon reception of this message, 

.„■■!!! 

r% receiver 1 10 puts a time stamp in the received message. The receiver then puts 

IJ 2 

□ 20 another time stamp T- on the message inmiediately before sending the message back to 
the sender. When sender 100 receives the message, it records the receiving time . 
Using ,7] , for i = 1,2, the sender computes an estimate of the clock offset as 

between sender 100 and receiver 110, 

Let Q be the receiver's clock offset from the sender. That is, if at a given instant 
25 the receiver's clock shows time and the sender's clock shows time t^, then 0 = -r^. It 
proves convenient to denote the fixed delay from the sender to the receiver (called 
upward direction) by and the fixed delay from the receiver to the sender (called 
downward direction) by d^ . In this representation, the fixed delay includes all non- 
variable components of the delay such as transmission delay and propagation delay. 
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Let denote the variable component of the delay of the /-th message from the 
sender to the receiver and ef denote the variable component of delay from the receiver to 
the sender. The variable delay component is the part of the delay that would not occur 
under ideal conditions, and includes such delays as packet queuing delay and delay due to 
5 the unavailabiUty of shared resources (e.g., CPU or bandwidth). 
We then have the following equations 

7;'-7;' = j"+0+< (1) 

T^'-T^^=d^~9+ef (2) 
For ease of further description of present inventive algorithms, it proves 
10 convenient to define X^=Tl -T^ and =7;^ -T^ , where i = 1, n after the exchange 
of n messages. 

The above-cited SDEA keeps the smallest values of X. and , viz^, it keeps 
variables 



iU 15 and 



C/„^niin{ZJ, 

1=1 n 



The estimate for 6 using n samples is then computed as 

i = {U„-V^)l2. (3) 

Separate Direction Estimation Algorithm with Bias Correction (SDEABC) 
20 From (3), with E[-] as the expected value operator we can see that, assuming 

E[ej = E[U„-VJf2 

= 6) + (£[ min {<} ] - £[ min { } ] ) / 2 . (4) 

i=l,,..,n 1=1,.. .,n 

Therefore, the SDEA is asymptotically unbiased if the pdf of both e- > 0 and ef > Ois 
25 positive near 0. However, with a finite number of samples, the estimator is biased if 
^[minlef }1 ^ E[mm{ef }] , a not unlikely event because of differences between uplink 

i=U...,n i=l,. .,n 
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and downlink traffic loading. As will be seen, illustrative embodiments of the present 
invention reduce such errors in estimates due to bias. 

The empirical distribution of a random variable /? > 0 is illustratively constructed 
in a manner now to be described. Suppose R is independently sampled n times with the 
5 result being r;, . . . , r„. We rearrange the n samples so that r^.„ < r2.^ < ... < r^.„, where each 

rtn is from the original sample set. Further, let = 0 and rn+im - ^ . The empirical 
distribution of R is then 

«+l ; _ 1 

FAx) = 2i—)nrt-v.^^<rrJ. (5) 

where /(•) is an indicator function, ie., /(•) = 1 when the argument of / is satisfied, and is 
10 0 otherwise. Equivalently, the complementary distribution function is 

O F,(x) = 2i^^-^)Iir,.,„<x<r,J. (6) 

p Note that since the intervals [n.i^n. r^n) are non-overlapping, 

p-! 

13 15 Suppose that independent, identically distributed (i.i.d.) random variables 

P i=U2,,,,,n have a distribution function F( x ). Then, min"^i{/?.} has the complementary 

M 

0 distribution functions (l-F(x))". Define 

Yn^ElrmxvUR^]. 

By using the empirical distribution function Fn(x) to replace F(x), an estimate of yI , Yn ^ 
20 is obtained as 

A CO 

^J(l-F„(x)rdr 

0 

r}=|;(^^r*(^:„-^-xJ. (8) 

As shown in (4), the bias of the SDEA estimator is 
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A 

and an estimate of this bias, bn , is given by 
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^«-((rf"rI)/2)-0n. (9) 

Based on the foregoing, it can readily be seen that a more accurate determination 
can be made of clock offset by correcting for bias of estimates made using SDEA. That 
is, by determining analytically what the bias in an estimate using SDEA is in a particular 
environment, correcting by an amount equal to the bias achieves a more accurate estimate 
of clock offset. This more accurately estimated correction is then advantageously applied 
to the out-of-synchronization clock. Moreover, the expected value of the bias provides a 
monitor for the analytical process; variations of this expected value over time can signal 
conditions in a network that may indicate greater or lesser confidence in clock offset 
estimates. 

In accordance with an approach that is preferred for some applications, a method 



15 



again proceeds from evaluation of 9^ = (t/„ - V„ ) / 2 , as in Eq. (3). Since 

C/„^min{XJ, 



and 



Vjrmn{Y,}^ 



i=l,...,n 



forming the expected value of each side of Eq. (3) and employing the notation from Eq. 
(8) for each of random variables J7„ and Vn yields 



20 



E[ej = E[(U„-VJ]J2 



E[UJ 



E[VJ 



We may take 



y: 



(10) 
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but in forming an estimate of bn , the bias in the estimate of 9 after n samples, we 
employ 

A A X A y 

b„=-(Y^-Y)--(n>in{X,}-mn{Y,}). 

Then, in accordance with the present approach, a bias-corrected (BC) estimate 
5 (SDEABC) for d is given by 

A A A 

= (imn{Z,}-min{}^})-|(y^^-rp. 

FIG, 2 shows an illustrative single-server network deployment of present 
inventive techniques. Specifically, FIG. 2 shows a time server 200 connected through a 

10 network 210 to a pluraHty of other network nodes 220-/, / = 1, 2, , . N. Nodes 220-/ may 
be routers, switches, servers of various kinds, network end points (including terminals, 
workstations or computers), or any other kind of network node. Each of nodes 220-f has 
a clock and messaging facilities for exchanging messages with time server 200 in the 
manner described above. That is, time server 200 forms one of the pair of nodes and, in 

15 turn, one or more (typically all) of the nodes 220-/ forms the other of the node pair for 
purposes of exchanging time-stamped messages and deriving offset estimates and 
estimate bias information. While each of the nodes 220-/ may have equal access to time 
server 200, priorities may be accorded some nodes 220-/, or some nodes 220-/ may be 
accorded access to server 200 more frequently. 

20 By exchanging messages with nodes 220-i, time server 200 will provide clock 

offset estimates and estimate bias information as described above, which information is 
available at nodes 220-i for correcting clock offset. Of course, N may have a value of 1, 
so that only a single network node device may interact with a particular time server. 
While time server 200 is shown as a separate dedicated function network node, it will be 

25 understood that the function of network node 200 may be included in a node performing 
other functions. Likewise, many network arrangements will have a plurality of time 
servers, each serving network nodes connected on a respective network or sub-network 
210. 
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FIG. 3 shows an illustrative alternative network arrangement in which a plurality 
of time servers 330 and 340-/, / = 1, 2, . . M, are connected in hierarchical relation 
through a plurality of networks 3 10- /, z = 1 , 2, . . . , M. In the illustrative arrangement of 
FIG. 3, only two levels are shown in the server hierarchy, but those skilled in the art will 
5 recognize that any number of levels of time servers may be used. Likewise, while the 
number of networks is shown equal to the number of nodes at the lowest hierarchical 
level, no such limitation is required in practicing the present invention using a 
hierarchical arrangement of time servers. Each of the networks 310-/ has one or more 
network nodes capable of accessing the respective time server connected to the network. 
10 By way of illustration, network 310-1 has nodes 350-1 1 through 350-lP connected to it. 
Likewise, network 310-M is shown having nodes 350-Ml through 350-MQ. Here, P and 
Q may be any integer. 

In operation, time server 330 exchanges time-stamped messages with each of the 
"^'^ time servers 340-i to provide the latter with offset estimates and estimate bias information 

p 15 of the type illustrated above to permit clock correction at the illustrative (second-level) 
}|j time servers 340-i. Each of the time servers 340-i then serves the clock correction 

RJ requirements of respective nodes 350-xx in the same manner. Of course, when more than 

Q two hierarchical levels of time servers are used, each level (after the first or highest) 

5 derives clock synchronization information from a time server at the next highest level 

III 20 The number of nodes will generally vary from one network 310-i to another, and all or 
|£ some of networks 310-i may be sub-networks of a larger network. Some time servers 

may be connected to nodes such as 350-xx and to a next lower order node as well. Some 
or all time servers may be located in the same local area or distributed over a wide area 
(including globally) to meet load and geographic distribution requirements for clock 
25 synchronization service. 

Access to respective time servers by particular nodes (or subordinate time servers) 
may be scheduled (e.g., periodic), dependent upon availability of time server resources, 
dependent on prior clock offset behavior at particular nodes (or subsidiary time servers) 
or detected conditions at such nodes or subsidiary time servers. Exchange of messages 
30 and derivation of correction information in accordance with present inventive teachings 
may be initiated, in appropriate cases, by a particular time server or by a node (or 
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subsidiary time server) for which the particular time server provides clock 
synchronization services. In each case, however, the time server acts as the reference 
source of time in determining offset using exchanges of time-stamped messages of the 
types described above. Thus, for example, a particular node (routinely, or in response to 

5 conditions detected at that node) may request that an associated time server initiate a 
synchronization sequence and supply the results to the particular node. 
Numerous and varied particular applications of the present inventive principles, all within 
the spirit of the present description and scope of the attached claims, will prove useful to 
those skilled in the art. 

10 What is claimed is: 
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